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FOREWORD 


This  document  contains  an  introduction  to  some  of  the  test  methods  currently  being  de- 
veloped by  the  Testing  Project  of  the  IGES/PDES  Organization.  This  is  a voluntary  or- 
ganization which  is  coordinated  by  the  United  States  National  Institute  of  Standards  and 
Technology  (NIST),  formerly  the  National  Bureau  of  Standards  (NBS). 

Readers  must  be  aware  that  this  area  of  technology  is  developing  rapidly  and  that  the  infor- 
mation here  reflects  the  “current  understanding.'1  Later  versions  of  this  document  may  have 
substantial  differences.  Full  consensus  has  not  been  reached  on  all  details. 

This  document  has  been  created  and  distributed  only  as  an  aid  to  understanding  the  topic  of 
testing  product  data  exchange  standards  and  software,  and  the  participating  organizations 
involved  within  the  USA.  It  should  not  be  used  as  a definitive  source  of  technical  information 
or  for  procurement  purposes. 

Suggestions  for  additions  and  corrections  to  this  document  should  be  sent  to  the  IGES/PDES 
Testing  Project  Manager.  Contact  details  axe  given  in  Section  9. 
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1 Introduction 


1.1  Purpose 


This  document  describes  the  structure  of  the  IGES/PDES  Testing  Project,  and  some  of  the 
techniques  and  methodologies  that  it  is  investigating  and  developing. 

Each  section  in  this  document  provides  an  outline  of  a particular  committee  and  gives  a 
summary  of  the  work  that  it  performs.  Readers  are  encouraged  to  consult  the  detailed 
documents  created  by  each  of  these  technical  committees  (these  are  referenced  within  the 
appropriate  sections). 


1.2  The  need  for  testing 

Experience  has  shown  that  data  exchange  translation  software,  based  on  IGES  or  other 
exchange  specifications/standards,  does  not  always  perform  as  the  user  expects.  The  problem 
may  be  made  up  of  several  components  including  : 


1.  Restrictions  or  errors  within  the  translation  software 

2.  Use  of  techniques  within  the  translation  software  which  are  inappropriate  for  a partic- 
ular user  or  application 

3.  Inaccuracies  or  ambiguities  within  the  data  exchange  specification  itself 

4.  Incompatibilities  between  CAD/CAM/CAE  1 systems 

5.  Unrealistic  user  expectations 


All  of  these  issues  need  to  be  addressed  if  data  exchange  is  to  be  made  predictable  and  reliable, 
and  it  is  by  generating  information  about  the  way  that  the  translation  software  works  (or 
fails)  that  testing  can  help  most. 

The  ultimate  2dm  of  all  data  exchange  specifications  and  software  packages  is  to  exchange 
actual  production  data  between  industrial  enterprises.  Therefore  it  is  vital  that  users  are 
able  to  test  that  this  aim  can  be  achieved  within  their  working  environment.  Testing  tools 

^or  the  rest  of  this  document  CAE  is  used  to  indicate  the  totality  of  CAD/CAM/CIM/CAE/FEA.  See 
Glossary. 
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and  methodologies  are  required  to  help  users  ensure  that  useful  and  accurate  data  can  be 
transferred. 

Many  types  and  levels  of  testing  are  possible,  all  have  advantages  and  disadvantages,  but 
it  is  possible  to  combine  the  various  elements  to  cover  a wide  range  of  requirements.  As 
a minimum  requirement,  a user  must  be  confident  that  any  data  he  wishes  to  transfer  can 
be  converted  into  and  out  of  a neutral  format,  such  as  IGES,  in  a consistent  and  accurate 
manner. 

A first  level  of  useful  testing  involves  measuring  the  behaviour  of  the  translation  software 
against  some  documented  criteria,  such  as  syntax  rules  and  conversion  rules  (or  mappings). 
Such  testing  begins  to  establish  how  the  implementor  has  chosen  to  convert  CAE  system 
features  into  IGES  constructs,  and  vice-versa.  This  approach  can  be  used  by  industrial  users, 
or  a small  number  of  testing  agencies  might  be  established  to  perform  the  tests  and  publish 
the  results.  The  independent  agency  approach  avoids  the  need  for  each  user  to  perform  the 
same  basic  tests  on  a piece  of  software. 

The  implementors  may  also  find  that  independent  testing  of  their  products  can  be  valu- 
able. Independent  testing  can  provide  addition  quality  assurance,  and  can  avoid  performing 
repeated  benchmark  tests  for  potential  purchasers. 

To  avoid  each  user  repeating  tests  on  processors,  for  application  specific  requirements,  it  is 
possible  to  develop  rigorous  definitions  of  the  way  that  particular  application  information  is 
to  be  transferred.  This  allows  more  sophisticated  and  thorough  testing  to  be  performed,  and 
thus  gives  a much  higher  level  of  confidence  in  the  quality  and  appropriateness  of  translation 
packages.  Testing  at  this  level  will  begin  to  guarantee  that  all  information  within  a particular 
domain  can  be  transferred  between  systems. 

Thorough  testing  can  increase  the  quality  of  translator  software,  it  can  generate  useful  feed- 
back to  the  standards  makers,  help  to  educate  users,  and  it  can  increase  user  confidence 
in  data  exchange.  All  of  these  factors  will  combine  to  make  industry  more  efficient  and 
competitive  by  providing  reliable  digital  based  operations. 


1.3  Charter  of  the  IGES/PDES  Testing  Project 

The  IGES/PDES  Testing  Project  was  established  to  create  and  maintain  a collection  of  test 
methods,  data  sets  and  software  tools  in  order  to  support  a variety  of  types  of  testing  on  IGES 
and  STEP/PDES  processors  and  data  files.  This  charge  includes  coordinating  the  work  of 
the  project  with  that  of  other  individuals  and  groups  engaged  in  similar  efforts. 
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1.4  Organization  of  the  IGES/PDES  Testing  Project 


The  IGES/PDES  Testing  Project  comprises  five  technical  committees.  These  are  : 


1.  Verification  Testing  Methodology  (VTM)  Committee 

2.  Application  Validation  Methodology  (AVM)  Committee 

3.  Interoperability  Testing  Methodology  (ITM)  Committee 

4.  Test  Case  Development  (TCD)  Committee 

5.  Testing  Methodologies  (TM)  Committee 


These  five  committees  are  coordinated  by  the  IGES/PDES  Testing  Project  Manager,  who 
reports  to  the  Technical  Planning  Committee  and  the  Chairman  of  the  IGES/PDES  Orga- 
nization. 

In  addition  there  is  the  IGES/PDES  Testing  Project  Editorial  Committee,  this  comprises  the 
chairs  of  the  above  technical  committees  plus  the  IGES/PDES  Testing  Project  Manager.  It  is 
responsible  for  ensuring  editorial  and  technical  consistency  of  all  output  from  the  IGES/PDES 
Testing  Project,  prior  to  submission  to  the  IGES/PDES  Edit  Committee. 

The  National  Institute  of  Standards  and  Technology  (NIST)  maintains  a special  working 
relationship  with  the  testing  committees.  Unlike  other  aspects  of  IGES/PDES  work,  some  of 
the  testing  activities  require  startup  funding,  or  full-time  effort,  and  thus  all  aspects  cannot 
be  undertaken  by  the  voluntary  IGES/PDES  Organization.  For  example  NIST  is  helping  to 
establish  an  IGES  Test  Case  Library. 


1.5  Plans  of  the  IGES/PDES  Testing  Project 


Currently,  a substantial  portion  of  the  IGES/PDES  Testing  Project  effort  is  directed  towards 
providing  the  technical  resources  for  a National  IGES  Verification  Testing  Program.  These 
resources  have  been  developed  and  are  now  in  a beta  test  phase.  Following  completion  of 
the  beta  tests,  the  methodology  will  be  refined  and  additional  test  cases  will  be  included  to 
produce  a more  comprehensive  testing  program  for  IGES  3.0  and  4.0  translators. 

Another  large  effort  is  aimed  at  developing  and  approving  application  protocols  and  appro- 
priate testing  techniques.  This  work  includes  the  production  of  documentation  to  help  imple- 
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mentors  write  IGES/PDES  application-based  translators  and  the  development  of  a testing 
program  for  application-based  translators. 

Users  of  IGES  processors  will  be  supported  with  guidelines  for  carrying  out  their  own  types 
of  testing,  and  case  studies  of  such  exercises  will  be  published  to  illustrate  data  exchange  in 
operation.  Additional  test  methods  will  be  investigated  for  IGES  processors,  and  methods 
will  be  established  for  STEP/PDES. 

There  are  various  other  groups  working  on  the  testing  of  data  exchange  standards  and  related 
technologies.  It  is  the  intention  of  the  IGES/PDES  Testing  Project  to  cooperate  with  such 
groups  and  to  exchange  applicable  ideas. 

Many  aspects  of  this  work  will  be  outlined  in  future  versions  of  this  overview  document,  and 
detailed  in  technical  committee  documents. 
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Verification  Testing 


2.1  Introduction 


To  provide  data  on  quality  and  usefulness  of  IGES  processors,  the  IGES/PDES  Organization 
and  NIST  are  establishing  the  National  IGES  Verification  Testing  Program.  This  program 
is  based  upon  the  methodology  described  briefly  here  and  detailed  in  the  documents  of  the 
Verification  Testing  Methodology  Committee,  in  particular  reference  [l].  The  goal  of  this 
verification  testing  is  the  substantiation,  by  an  independent  agency,  of  an  implementor's 
claims  for  the  entity  mappings  and  other  processing  performed  by  an  IGES  processor. 


2.2  Verification  testing  methodology 


The  methodology  consists  of  gathering  from  the  client,  for  each  IGES  processor  being  tested, 
a set  of  claims  for  entity  mappings  and  other  processing.  A formal  test  plan  is  developed  to 
ensure  that  sufficient  testing  is  conducted  to  verify  the  claims  against  observed  outcome.  This 
demonstrates  that  each  claimed  mapping  is  correctly  described  and  constructed  in  accordance 
with  the  IGES  specification. 

This  approach  is  designed  to  provide  client  and  users  wdth  information  about  what  to  expect 
from  a single  given  IGES  processor  (pre  or  postprocessor)  in  isolation  and  to  provide  the 
foundation  upon  which  other  testing  activities  can  build.  It  will  also  provide  vendors  with 
independently  derived,  concrete  data  about  their  products  with  which  to  answer  customer 
questions. 


2.3  The  role  of  a Testing  Agency 


Verification  tests  are  administered  by  one  or  more  independent,  third  party  testing  agen- 
cies. These  agencies  implement  the  verification  testing  methodology  as  developed  by  the 
IGES/PDES  Organization. 

Reference  [l]  provides  detailed  descriptions,  but  briefly,  the  methodology  describes  four  func- 
tional areas  that  a testing  agency  must  implement  : 


The  Review  Board:  This  is  a corporate  component  of  the  testing  agency  which  serves  as 
an  overseer  of  the  agency’s  entire  verification  testing  enterprise.  The  board  reviews 
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and  approves  the  qualifications  and  procedures  of  the  IGES  Verification  Panel  and  any 
testing  laboratories  used,  sets  fees,  and  ensures  that  the  rules  of  the  agency  are  followed. 

Administrative  Staff:  The  people  with  documented  procedures  who  provide  logistic  and 
administrative  support  for  the  testing  enterprise  of  the  agency.  Responsibilities  include 
scheduling  meetings  and  tests,  collecting  fees,  keeping  records,  and  ensuring  insurance 
coverage. 

IGES  Verification  Panel:  This  panel,  selected  by  the  test  agency  review  board,  consists 
of  experts  in  IGES  and  CAE  data  exchange.  Panel  members  must  act  independently, 
rather  than  as  representatives  of  their  professional  affiliation.  Panel  responsibilities 
include  formulating  operating  procedures,  reviewing  verification  requests,  writing  test 
plans  and  issuing  formal  summary  reports  of  test  outcomes. 

Testing  Laboratory:  The  testing  laboratory  actually  performs  the  test  in  accordance  with 
a test  plan  provided  to  it  by  the  IGES  Verification  Panel.  Responsibilities  of  the  Testing 
Laboratory  include  establishing  the  test  environment,  executing  the  test  cases  specified 
and  documenting  results  in  accordance  with  the  methodology  of  reference  [1].  There 
may  be  more  than  one  testing  laboratory  employed  by  the  testing  agency. 


The  flow  of  the  verification  process  is  depicted  in  Figure  1.  It  begins  when  an  implementor, 
vendor,  or  other  agency,  called  the  “client,”  submits  a completed  verification  request  package. 
The  completed  package  contains  the  client’s  claims  for  entity  mappings  between  native  entities 
and  IGES  entities  (or  vice  versa)  for  a single  IGES  processor.  This  information  and  system- 
specific  details  for  setup  and  operation  of  the  implementation  under  test  form  the  basis  for 
both  the  test  plan  and  later  analysis  of  the  results. 

After  receiving  the  request,  the  IGES  Verification  Panel  reviews  it  for  completeness  and  writes 
a test  plan  utilizing  test  cases  available  in  the  IGES  Test  Case  Library.  Testing  agency  staff 
then  select  a test  laboratory,  schedule  the  tests,  and  forward  the  test  plan  to  the  selected 
laboratory.  The  testing  laboratory  obtains  the  required  test  cases  from  the  library,  runs  the 
prescribed  tests  according  to  the  test  plan,  gathers  the  results,  and  forwards  the  results  file 
to  the  IGES  Verification  Panel  for  analysis. 

After  completing  the  required  analysis  and  formulating  its  conclusions,  the  IGES  Verification 
Panel  issues  a summary  report  listing  those  claims  that  were  verified  and  those  that  were 
not.  The  panel  then  provides  a copy  of  the  results  file  and  the  summary  report  to  the  client 
for  review  prior  to  any  release  of  the  data.  The  client  has  an  opportunity  (within  30  days)  to 
respond,  in  writing,  to  my  perceived  problems.  Problems  are  resolved  by  negotiation  and/or 
retest. 
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Figure  1:  The  flow  of  the  verification  process 
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2.4  Privileges  and  obligations  of  clients  and  users 


It  is  important  to  the  credibility  and  the  usefulness  of  the  National  IGES  Verification  Testing 
Program  that  clients  represent  their  support  of  IGES  and  their  participation  in  IGES  testing 
appropriately.  Completing  the  verification  program  in  no  way  implies  the  approval  of  an  IGES 
processor  by  the  IGES/PDES  Organization,  the  testing  agency,  or  the  National  Institute  of 
Standards  and  Technology.  In  addition,  participation  in  the  beta  testing  of  the  verification 
methodology  does  not  constitute  having  been  verified  and  under  no  circumstances  should  be 
so  claimed  or  implied. 

There  will  be  a mechanism  for  investigating  any  discrepancies  discovered  in  IGES  processors 
which  have  been  submitted  to  the  National  IGES  Verification  Testing  Program. 
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3 Application  Validation  Testing 


3.1  Introduction 


The  Application  Validation  Methodology  (AVM)  Committee  is  responsible  for  developing 
and  documenting  procedures  for  ensuring  that  information  can  be  completely  and  reliably 
exchanged  within  a specific  application  area.  This  committee  provides  guidelines  for  the 
development  and  validation  of  IGES/PDES  application  protocols  and  procedures  for  testing 
individual  processors  for  conformance  to  these  application  protocols. 

The  AVM  Committee  primary  work  items  are: 


1.  Define  the  required  content  of  an  application  protocol  (AP)  and  the  procedures  for 
testing  and  approving  APs 

2.  Review  draft  APs  at  the  request  of  the  submitting  IGES/PDES  technical  committees 

3.  Develop  and  document  standard  methods  for  testing  conformance  of  a processor  to  an 
AP 

4.  Develop  and  document  guidelines  on  the  use  of  APs 


The  AVM  Committee  is  also  working  with  its  counterparts  in  ISO  TC184/SC4/WG1  to 
develop  the  AP  methodology  in  conjunction  with  the  development  of  the  STEP  standard. 


3.2  Application  protocol  methodology 


The  concept  of  application  protocols  provides  a formal  procedure  for  specifying  neutral, 
application-specific  formats.  The  process  for  developing  an  AP  includes  identifying  the  in- 
formation requirements  of  an  application  domain  and  documenting  them  in  conjunction  with 
an  information  model  of  that  domain.  The  information  model  is  then  used  in  the  selection 
of  data  constructs,  from  IGES  or  STEP/PDES,  for  representing  the  required  information. 

The  key  components  of  an  AP  are: 


1.  A scope  and  requirements  statement  of  the  application 

2.  An  application  reference  model  that  describes  the  application’s  information  structures 
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3.  An  application  interpreted  model  that  specifies  the  IGES  or  STEP/PDES  constructs 
to  be  used  to  represent  the  application  information 

4.  An  AP  format  specification  including  a usage  guide 

5.  A collection  of  conformance  requirements  and  test  cases 


Currently  there  are  draft  IGES  APs  being  developed  as  prototypes  for  testing  and  refin- 
ing these  concepts.  The  STEP/PDES  community  has  recently  initiated  parallel  efforts  for 
applying  the  AP  methods  to  that  future  standard. 


3.3  Application  protocol  approval  process 


Application  protocols  are  developed  by  a formal  process  that  begins  with  the  selection  of 
candidate  application  areas.  The  complete  process  is  best  performed  by  a group  of  application 
experts,  and  is  not  covered  in  this  document.  Those  who  are  interested  in  these  specifics  are 
directed  to  the  AVM  Committee  document  “Guidelines  for  the  Specification  and  Validation 
of  IGES  Application  Protocols”  reference  [2]. 

Once  a candidate  AP  is  complete  and  has  been  successfully  validated  (as  specified  in  reference 
[2]),  it  must  be  approved  by  the  AVM  Committee.  With  this  approval,  the  candidate  AP  will 
be  presented  to  the  Edit  Committee  of  the  IGES/PDES  Organization  for  final  approval. 


3.4  Conformance  testing  of  application  protocol  processors 


Because  an  approved  AP  should  contain  explicit  requirements  for  information  representations 
and  conversions,  it  is  possible  to  perform  very  rigorous  conformance  testing  upon  any  transla- 
tion software  which  claims  AP  conformity.  This  requires  that  the  full  functionality  specified 
by  the  application  reference  model  is  exercised,  and  that  none  of  the  format  specification 
rules  are  broken. 
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4 Interoperability  Testing 


4.1  Introduction 


The  Interoperability  Testing  (ITM)  Committee  is  responsible  for  developing  and/or  identify- 
ing documents,  methodologies,  and  tools  for  ensuring  that  the  required  level  of  information 
can  be  adequately  exchanged  between  CAE  systems  within  a specific  user  environment.  This 
committee  provides  guidelines  for  the  development  of  user's  requirements  and  provides  pro- 
cedures for  testing  the  transfer  of  production  data  in  accordance  with  these  requirements. 

Typical  ITM  committee  work  items  are  : 


1.  Updating  and  maintaining  an  interoperability  testing  methodology  document. 

2.  Documenting  case  studies 

3.  Developing  user  guides  on  specific  topics 

4.  Maintaining  a listing  and  brief  description  of  software  tools  which  might  be  useful  for 
interoperability  testing 

4.2  Interoperability  test  methodology 


An  interoperability  test  is  built  upon  known  information  regarding  the  specific  IGES  proces- 
sor implementations  involved  in  the  exchange.  Such  information  could  well  be  gained  from 
the  National  IGES  Verification  Program  and  application  protocol  based  testing.  The  inter- 
operability test  is  conducted  within  the  user’s  environment  and  is  evaluated  based  upon  the 
user’s  specific  requirements.  The  evaluation  determines  what  effect  the  imperfections  of  each 
implementation  have  on  the  overall  translation.  Depending  on  the  user’s  requirements,  some 
imperfections  may  be  tolerable. 

The  key  activities  of  an  interoperability  test  are  : 


1.  Establishing  the  user’s  data  exchange  requirements 

2.  Acquiring  and  analyzing  an  entity  map  comparison  of  the  systems  involved  before  spec- 
ifying the  scope  of  testing 

3.  Selection  or  development  of  appropriate  test  cases 
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4.  Conducting  the  test 

5.  Analyzing  the  test  results  for  acceptability  for  the  specified  task 

6.  For  unacceptable  test  outcomes,  identifying  alternative  mechanisms  to  achieve  an  ac- 
ceptable outcome 

The  results  of  an  interoperability  test  can  provide  the  user  of  IGES  processors  with  : 


1.  Increased  confidence  in  the  translation  process  between  the  specified  CAE  systems 

2.  An  indication  of  the  quality  and  completeness  to  be  expected  from  a translation  before 
serious  costs  have  been  incurred 

3.  An  awareness  of  the  capability  and  limitations  of  particular  IGES  processor  implemen- 
tations 

4.  The  ability  to  identify  or  avoid  potential  translation  problems  with  particular  IGES 
processor  implementations 


In  summary,  interoperability  tests  assist  the  user  in  having  more  controlled  and  effective  data 
exchange. 


12 


5 Test  Case  Development 


5.1  Introduction 

The  Test  Case  Development  (TCD)  Committee  is  responsible  for  developing  test  cases  and 
documentation  for  use  by  the  IGES/PDES  Testing  Project,  the  National  IGES  Verification 
Testing  Program,  and  for  use  by  individuals  wishing  to  test  their  own  IGES  processor  soft- 
ware. 

Those  test  cases  developed  by  the  TCD  Committee  will  be  reviewed  by  the  IGES/PDES 
Organization  and  then  added  to  the  official  IGES  Test  Case  Library. 

Typical  TCD  Committee  work  items  are: 

1.  Providing  the  test  cases  for  use  in  the  beta- testing  phase  of  the  National  IGES  Verifi- 
cation Testing  Program 

2.  Developing  and  maintaining  a guide  for  IGES  Test  Case  Development 

3.  Developing  guidelines  and  test  cases  for  application  protocol  and  interoperability  testing 

4.  Developing  a methodology  for  specifying  test  case  requirements,  and  specifying  a soft- 
ware tool  to  aid  test  case  specification 

5.  Documenting  guidelines  for  writing  the  next  generation  of  test  cases  based  on  the  test 
case  specification  methodology. 


As  test  case  development  guides  are  completed,  it  is  anticipated  that  each  technical  committee 
responsible  for  revising  or  adding  a new  feature  to  the  IGES  specification,  will  use  these 
guides  to  develop  test  cases  for  that  feature.  By  doing  this,  test  cases  for  new  features  will 
be  available  to  the  IGES  community  for  processor  testing. 

Currently  all  test  case  development  activity  is  based  on  IGES  versions  3.0  and  4.0.  No  test 
cases  are  being  considered  for  earlier  versions  of  IGES. 


5.2  Test  case  development  methodology 

The  process  of  developing  a test  case  includes  identifying  the  scope  of  the  test  to  be  per- 
formed, determining  the  testing  criteria,  identifying  the  entities  required,  generating  a test 
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specification,  and  writing  a test  case  to  satisfy  the  specific  requirements. 

The  preprocessor  portion  of  the  test  case  takes  the  form  of  a generic  script  describing  con- 
struction details  and  supporting  documentation.  Generic  instructions  ensure  test  case  inde- 
pendence, avoidance  of  software  specific  terminology,  and  preservation  of  proprietary  rights 
of  CAE  systems. 

Each  test  case  contains  : 


1.  A unique  identifier 

2.  A liability  statement 

3.  A list  of  entities  required  for  the  test 

4.  A stated  purpose  for  the  test 

5.  A description  of  data  used  within  the  test 

6.  A script  and  results  expected  for  the  preprocessor 

7.  An  IGES  file  and  results  expected  for  the  postprocessor 

8.  A pictorial  representation  of  the  test  data 

The  only  information  not  resident  within  the  test  case  is  the  system  environment  setting. 
Test  cases  are  structured  so  that  preprocessor  and  postprocessor  are  tested  independently.  A 
complete  description  of  test  case  structure  is  contained  in  reference  [4]. 


5.3  Test  case  review 


When  a test  case  has  been  written,  either  by  members  of  the  TCD  Committee  or  a IGES/PDES 
technical  committee,  a review  of  the  test  case  begins.  At  least  three  software  tools  analyze 
the  test  case  for  syntax  accuracy  and  file  integrity.  All  errors  are  corrected. 

Next  the  test  case  is  reviewed  for  completeness,  consistency,  and  compliance  with  the  test 
case  specification.  The  test  part  must  be  created  on  three  different  CAE  systems  by  three 
different  proficient  CAE  operators.  All  problem  areas  are  recorded  throughout  the  review. 
In  addition,  each  reviewer  provides  recommendations  for  improving  test  case  instructions. 
Comments  a le  evaluated  by  the  TCD  Committee  and  the  test  case  modified  accordingly. 
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Finally,  each  major  release  of  test  cases  is  brought  before  the  Testing  Project  Editorial  Com- 
mittee. If  there  are  no  technical  problems  identified,  the  test  cases  are  approved  and  sent  to 
the  Test  Case  Librarian  for  insertion  into  the  official  IGES  Test  Case  Library. 


5.4  Test  case  coverage 


Test  cases  developed  for  verification  testing  will  be  single  entity,  for  testing  a single  IGES 
processor  at  a time.  Single  entity  is  defined  as  being  a primary  entity  under  test  but  support- 
ing entities  may  have  to  be  present  to  create  a valid  environment.  For  example  an  angular 
dimension  entity  requires  witness  line,  annotation,  and  leader  entities. 

A suite  of  test  cases  are  being  developed  for  the  Verification  Testing  Methodology  Committee, 
to  support  the  National  IGES  Verification  Testing  Program  as  described  in  reference  [l] . 

Future  test  cases  will  address  attribute  and  functionality  testing.  Test  cases  to  support 
application  validation  testing  as  stated  in  reference  [2]  will  be  based  upon  on  this  work. 
Test  cases  for  the  Interoperability  Testing  Methodology  Committee  will  be  highly  specialized 
and  directed  to  a specific  data  exchange  problem,  but  the  methodology,  structures  and  tools 
developed  by  the  TCD  committee  should  be  useful  and  appropriate. 

It  is  important  that  test  cases  are  developed  so  that  the  different  levels  of  testing  described 
in  this  document  can  build  one  upon  the  other.  That  is  single  entity  testing  must  be  done 
before  multiple  entity  testing  is  attempted.  Likewise  single  processor  testing  must  be  done 
before  interoperability  testing  is  attempted. 


5.5  IGES  Test  Case  Library 

An  official  IGES  Test  Case  Library  is  being  built  as  a co-operative  effort  with  the  TCD 
Committee,  IGES/PDES  Organization,  NIST,  other  Government  agencies,  and  industry. 

The  official  IGES  Test  Case  Library  will  reside  at  NIST  and  be  placed  in  the  public  domain. 
It  is  the  intent  of  the  IGES/PDES  Testing  Project  that  all  approved  test  cases  be  available 
for  use  by  anyone  to  perform  testing  on  IGES  processors.  As  the  IGES/PDES  Testing 
Project  expands, the  library  will  become  more  extensive  and  will  contain  very  sophisticated 
and  complex  test  cases. 

Questions  regarding  the  library  of  test  cases  should  be  directed  to  the  Test  Case  Librarian. 
(Contact  details  are  given  in  section  9.) 
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6 Testing  Methodologies 


6.1  Introduction 


The  Testing  Methodologies  Committee  has  the  responsibility  for  collecting  and  disseminating 
information  that  might  be  appropriate  or  applicable  to  the  rest  of  the  IGES/PDES  testing 
community. 


6.2  Alternative  methodologies 


There  has  been  significant  progress  in  the  testing  of  information  technology  standards  and 
specifications  such  as  GKS,  ISO/OSI  Networking,  and  various  compilers  such  as  Pascal  and 
Ada.  Although  none  of  these  is  identical  to  IGES  or  STEP/PDES  there  axe  some  similari- 
ties. Several  of  these  standards  have  accredited  testing  services  available,  which  means  that 
formal  procedures  have  been  developed,  and  have  been  accepted  by  some  national  laboratory 
accreditation  scheme. 

In  addition  there  has  been  significant  work  on  the  testing  of  data  exchange  specifications  and 
standards  such  as  SET,  VDA-FS,  and  IGES  within  Europe.  The  finding  from  such  work  can 
be  fed  into  the  IGES/PDES  arena  through  this  committee  to  take  full  advantage  of  applicable 
ideas. 

Conformance  testing  of  STEP  processors  will  require  much  more  sophistication  than  the 
current  IGES  processor  testing  methodologies.  This  committee  is  responsible  for  identifying 
significant  concepts  and  contributions. 
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7 Summary 


The  IGES/PDES  Testing  Project  is  addressing  the  problems  of  testing  the  processes  and 
tools  of  data  exchange  in  a variety  of  ways.  The  most  stable  of  these  alternatives  are  outlined 
in  this  document,  but  there  are  many  other  possibilities  which  have  been,  and  will  continue 
to  be  investigated. 

The  technical  work  towards  a National  IGES  Verification  Testing  Program,  is  well  advanced, 
but  the  administrative,  financial,  and  legal  aspects  are  outside  the  scope  of  the  IGES/PDES 
Testing  Project.  A series  of  documents  and  case  studies  is  being  prepared  to  aid  the  user 
with  in-house  testing. 

Work  on  developing  and  defining  sufficiently  rigorous  rules  for  application-  specific  processors, 
is  beginning  to  demonstrate  that  application  protocols  can  provide  a workable  solution  for 
IGES.  In  addition  this  approach  offers  a practical  approach  for  STEP/PDES  processors. 

Carefully  designed  and  constructed  test  cases  form  a common  link  between  all  testing  method- 
ologies, and  the  emphasis  is  moving  towards  providing  software  tools  to  generate  test  cases, 
as  being  the  only  realistic  solution  to  the  demand  for  extensive  test  case  libraries. 

The  work  being  carried  out  in  related  fields,  such  as  compiler  and  network  protocol  testing, 
by  various  national  and  international  bodies  is  being  investigated.  The  collection,  assimila- 
tion,and  dissemination  of  such  information  is  seen  as  an  important  task  for  the  IGES/PDES 
Testing  Project. 

As  more  work  is  done,  this  document  will  be  updated  on  a regular  basis.  To  maintain  a broad 
view  of  the  variety  of  work  in  this  field,  it  will  lead  through  to  the  more  detailed  documents 
of  the  IGES/PDES  Testing  Project  and  will  act  as  part  of  the  information  dissemination 
process. 
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8 Terminology 


This  section  defines  the  terminology  for  all  committees  and  documents  of  the  IGES/PDES 
Testing  Project;  consequently  it  includes  some  terms  which  do  not  appear  within  this  doc- 
ument. Some  terms  are  not  yet  in  common  usage  within  the  IGES/PDES  Testing  Project; 
they  appear  here  so  that  a single  term  may  be  adopted  by  all  parties. 


8.1  Glossary  of  Terms 


The  intention  is  to  ensure  that  the  terms  used  within  the  Testing  Project  have  minimum 
conflict  with  the  terminology  being  used  in  other  related  fields  of  software  testing.  To  this 
end,  the  Draft  Glossary  being  defined  by  the  nearest  equivalent  ISO  committee  has  been 
used  as  a basis,  reference  [5].  However  it  should  be  appreciated  that  currently  there  are  some 
conflicts  in  usage,  and  these  are  marked.  The  conflicts  will  be  resolved  and  documented,  but 
users  of  these  terms  should  be  aware  of  the  possibility  of  confusion  or  misunderstanding. 


ACCEPTANCE  TESTING  Formal  testing  conducted  to  determine  whether  a software 
system  satisfies  its  acceptance  criteria,  and  to  enable  the  customer  to  determine  whether 
to  accept  the  system  for  use  in  a specific  environment.  Formal  testing  includes  the 
planning  and  execution  of  several  kinds  of  tests  (e.  g.  functional,  volume,  performance) 
to  demonstrate  that  the  implemented  software  satisfies  the  customer  requirements  for 
the  software  system.  (Also  known  as  Uuser- acceptance  testing”.) 

ACCEPTANCE  TEST  MODEL  A test  piece  which  resides  on  either  the  sending  or  re- 
ceiving system  in  that  system’s  native  data  form,  used  in  Acceptance  testing. 

(LABORATORY)  ACCREDITATION  The  formalized  initial  and  continuing  process  of 
ensuring  a testing  laboratory  is  competent  to  carry  out  specific  types  of  tests. 

NOTE:  the  term  “laboratory  accreditation”  may  cover  the  recognition  of  both  the  tech- 
nical competence  and  the  impartiality  of  a testing  laboratory  or  only  its  technical  com- 
petence. Accreditation  is  normally  awarded  following  successful  laboratory  assessment 
and  is  followed  by  appropriate  surveillance. 

ACCREDITATION  BODY  A body  that  conducts  and  administers  a laboratory  accred- 
itation scheme  and  grants  accreditation. 

NOTE:  An  accreditation  body  may  wish  to  delegate  fully  or  partially  the  assessment 
of  a testing  laboratory  to  another  competent  body  (assessment  agency).  Whilst  it  is 
recognised  that  this  may  be  a practical  solution  to  extending  recognition  of  testing 
laboratories,  it  is  essential  that  such  assessment  be  equivalent  to  that  applied  by  the 
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accreditation  body  and  that  the  accreditation  body  takes  full  responsibility  for  such 
extended  accreditation. 

APPLICATION  INTERPRETED  MODEL  The  model  which  specifies  the  IGES  or 
STEP/PDES  constructs  to  be  used  to  represent  the  application  information. 

APPLICATION  PROTOCOL  A method  to  achieve  consistent  and  reliable  exchange  of 
definition  data  within  a specified  application  area.  The  key  components  of  an  appli- 
cation protocol  are  a conceptual  information  model  for  the  application  area  with  its 
supporting  documentation,  an  application  protocol  format  specification,  and  a set  of 
application  protocol  format  test  cases. 

APPLICATION  PROTOCOL  FORMAT  An  application  specific  format  that  is  based 
on  the  embedding  of  items  of  information  from  a conceptual  information  model  into 
specific  entities  of  a data  format. 

APPLICATION  PROTOCOL  FORMAT  SPECIFICATION  A specification  that  pro- 
vides a complete,  rigorously  defined  and  unambiguous  means  to  represent  the  informa- 
tion that  is  required  for  a specific  application  area;  consists  of  an  application  subset, 
the  restrictions  on  the  global,  directory  entry  and  parameter  data  sections  and  a usage 
guide  for  the  application  subset. 

APPLICATION  PROTOCOL  FORMAT  TRANSLATOR  An  application  specific  trans- 
lator that  is  based  on  the  embedding  of  CAD  information  from  the  application  protocol 
information  model  into  the  CAD  database  format  and  the  IGES  entities  in  an  applica- 
tion protocol  format.  The  translator  implements  a single  mapping  association  between 
a certain  entity  in  the  CAD  database  format  and  a certain  IGES  entity  (APF  prepro- 
cessor) and  between  a certain  IGES  entity  and  a certain  entity  in  the  CAD  database 
format  (APF  postprocessor)  to  satisfy  the  needs  of  one  application  protocol  and  its 
associated  application  protocol  format. 

APPLICATION  REFERENCE  MODEL  An  information  model  that  describes  the  in- 
formation requirements  and  the  information  structure  for  an  application  area.  The 
information  model  uses  application  specific  terminology  and  rules  familiar  to  cm  expert 
from  the  application  area.  The  model  is  independent  of  any  physical  implementation 
and  can  be  validated  by  an  expert  from  the  application  area. 

ATTRIBUTE  Information,  provided  in  specific  fields  within  the  directory  entry  of  an  IGES 
entity,  which  serves  to  qualify  the  entity  definition. 

CERTIFICATE  OF  CONFORMANCE  (OR  CONFORMITY)  A document  issued 
under  the  rules  of  a certification  system  indicating  that  adequate  confidence  is  provided 
that  an  IUT  in  in  conformance  with  a specific  standard  or  technical  specification  as 
determined  through  the  use  of  a specified  test  method. 
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CERTIFICATION  The  process  by  which  a product  or  service  is  awarded  a certificate  of 
conformance. 

CERTIFICATION  BODY  An  impartial  body  possessing  the  necessary  competence  and 
reliability  to  operate  or  accredit  operation  of  a certification  system,  and  in  which  the 
interests  of  all  parties  concerned  with  the  function  of  the  system  are  represented. 

CLIENT  The  organization  that  submits  a system  or  implementation  for  any  type  of  testing. 

CONFORMANCE  The  fulfillment  by  an  IUT  of  all  requirements  specified. 

CONFORMANCE  TESTING  The  testing  of  a candidate  product  for  the  existence  of 
specific  characteristics  required  by  a standard;  testing  the  extent  to  which  the  imple- 
mentation under  test  (IUT)  is  a conforming  implementation. 

DIRECTORY  ENTRY  SECTION  That  section  of  an  IGES  file,  consisting  of  fixed  field 
data  items  for  an  index  and  attribute  list  of  all  entities  in  the  file. 

ENTITY  The  basic  unit  of  information  in  an  IGES  file.  The  term  applied  to  single  items 
which  may  be  individual  elements  of  geometry,  collections  of  annotation  to  form  dimen- 
sions, or  collections  of  entities  to  form  structured  entities. 

ENTITY  TYPE  NUMBER  A positive  integer  used  to  designate  a specific  type  of  entity. 
For  example,  the  circular  arc  entity  has  an  IGES  entity  type  number  of  100. 

FALSIFICATION  TESTING  A test  method  developed  to  find  errors  in  the  implementa- 
tion. If  errors  are  found,  one  can  correctly  deduce  the  implementation  does  not  conform 
to  the  standard;  however,  the  absence  of  errors  does  not  necessarily  imply  the  converse. 
Falsification  testing  can  only  demonstrate  non-conformance. 

FLAVORING  A condition  that  exists  with  IGES  data  that  results  from  the  combined 
effects  of  differing  system  capabilities,  data  base  structures,  user  interfaces  and  trans- 
lator mappings  such  that  these  effects  results  in  a dispersal  of  information  content  into 
different  sets  of  IGES  entities. 

FORM  NUMBER  An  integer  which  is  used  when  needed  to  further  define  a specific  entity. 
This  becomes  necessary  when  there  axe  several  interpretations  of  an  entity  type. 

GEOMETRIC  Having  to  do  with  the  shape  information  (points,  curves,  surfaces  and  vol- 
umes) necessary  to  represent  an  object. 

GLOBAL  SECTION  That  section  of  an  IGES  file  consisting  of  general  information  de- 
scribing the  file,  the  file  generator  (preprocessor)  and  information  needed  by  the  file 
reader  (postprocessor). 

IGES  PROCESSOR  A generic  name  for  a software  package  used  to  translate  between 
CAD  system  and  IGES  ( and  vice  versa).  Sometimes  called  a translator. 
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IGES  VERIFICATION  PANEL  A group  of  CAD  industry  representatives  who  are  re- 
sponsible for  the  formal  testing  procedures  used  in  the  verification  process.  Other 
duties  include  the  creation  of  a test  plan,  production  of  final  report  and  the  arbitration 
of  disputes. 

IMPLEMENTATION  UNDER  TEST  (IUT)  That  part  of  a product  or  service  which 
is  to  be  studied  under  testing,  wrhich  should  be  an  implementation  of  one  or  more 
characteristics  of  the  standard 

INSTANCE  A particular  occurrence  of  some  item  or  relationship.  Several  instances  may 
reference  the  same  item. 

INTEROPERABILITY  TESTING  Related  to  acceptance  testing,  but  specifically  ap- 
plied to  the  examination  of  the  information  exchange  between  two  specific  IUTs  and  the 
ability  of  each  IUT  to  use  such  information.  This  does  not  form  part  of  conformance 
testing. 

MODEL  A particular  collection  of  data  which  describes  a product.  This  could  be  in  an 
IGES  file  or  within  a CAD  system. 

PARAMETER  DATA  SECTION  That  section  of  an  IGES  file  consisting  of  specific  ge- 
ometric or  annotative  information  about  the  entities  or  pointers  to  related  entities. 

POINTER  A number  that  indicates  the  location  of  an  entity  within  an  IGES  file. 

POSTPROCESSOR  tA  software  unit  that  translates  a file  of  product  from  the  form  of 
the  IGES  specification  into  the  native  data  base  form  of  a specific  CAD/CAM  system 

PREPROCESSOR  tA  software  unit  that  translates  a file  of  product  definition  data  from 
the  native  data  base  form  of  a specific  CAD/CAM  system  into  the  form  of  the  IGES 
standard. 

PRODUCT  DEFINITION  data  required  to  describe  and  communicate  the  characteris- 
tics of  physical  objects  as  manufactured  products. 

PROTOCOL  A set  of  conventions  or  rules  that  govern  the  interactions  of  processes  or 
applications  within  a computer  system  or  network. 

REVIEW  BOARD  A group  of  individuals  working  for  a testing  agency  who  are  responsible 
for  the  review  and  approval  of  the  testing  program  run  by  that  agency. 

START  SECTION  That  section  of  an  IGES  file  containing  a human- readable  prolog. 

SUBSET  A set  of  IGES  entities  that  is  less  than  the  set  of  all  IGES  entities  described  in 
the  IGES  specification. 
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SUMMARY  REPORT  document,  prepared  by  the  IGES  Verification  Panel,  describ- 
ing which  claims  about  a given  translator  could  be  verified  and  which  could  not.  A 
summary  of  a verification  test. 

SYSTEM  UNDER  TEST  (SUT)  The  computer  hardware,  software  and  communica- 
tions network  required  to  support  the  IUT 

TERMINATE  SECTION  The  final  section  of  an  IGES  file,  indicating  the  size  of  each  of 
the  preceeding  sections. 

TESTING  AGENCY  An  independent,  corporate  enterprise  that  is  responsible  for  the 
complete  execution  of  the  IGES  verification  test.  It  is  responsible  for  four  primary 
functions  which  are  : a review  board,  staff  function,  IGES  verification  panel  and  a 
testing  laboratory(ies). 

TESTING  LABORATORY  An  organization  that  conducts  an  IGES  verification  test  in 
accordance  with  a test  plan  submitted  by  a testing  agency. 

TEST  PLAN  A document  which  describes  sequences  of  tests  for  a specific  client.  The  test 
plan  is  derived  from  the  Verification  Request  Package  and  is  produced  by  the  IGES 
Verification  Panel. 

TEST  RESULTS  FOLDER  A folder  containing  the  output  of  a verification  test.  It  con- 
tains completed  forms,  the  test  log,  incident  reports,  and  all  hardcopy  generated  during 
the  test. 

TEST  SUITE  tA  collection  of  approved  test  cases. 

TRANSLATOR  see  IGES  processor 

(END)  USERS  Those  people  who,  in  response  to  specific  information  transfer  needs,  move 
CAD  data  between  similar  or  dissimilar  systems  and  validate  the  results  of  those  trans- 
fers. 

VALIDATION  *That  activity  which  assures  that  a product  or  process  functions  and  con- 
tains the  features  as  prescribed  by  its  requirements  and  specifications. 

NOTE:  can  also  be  defined  to  mean  the  conformance  assessment  process  and,  when 
conformity  is  demonstrated,  optionally  issuing  a certificate. 

VERIFICATION  *That  testing  of  an  implementor’s  claims  for  an  entity  mapping  and 
system  characteristics  to  assure  that  those  claims  are  accurate  and  that  the  translators 
correctly  produce  and  interpret  IGES  files  in  accordance  with  those  mappings  and  the 
IGES  specification,  (also  verification  testing) 

NOTE:  can  also  be  defined  to  mean  the  mathematical  proof  of  an  IUT’s  correctness, 
consistency  and  completeness. 
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VERSION  NUMBER  a means  for  uniquely  designating  one  specification  definition  or 
translator  implementation  from  a preceding  or  subsequent  one. 


8.2  Notes  on  usage  of  terms 


This  glossary  is  still  being  compiled,  and  some  of  the  terms  will  be  redefined  in  later  editions 
of  this  document.  In  particular  the  terms  marked  t or  t should  be  used  with  caution,  since 
alternative  applications  may  use  a similar  or  identical  term  with  a different  meaning. 

The  annotation  added  to  this  glossary  is  explained  below. 

f This  definition  is  a modified  form  of  an  ISO  definition  to  suit  the  application  to  IGES 
testing.  Alternatively  there  is  a very  similar  ISO  defined  term  which  might  be  confused. 

^ This  definition  conflicts  with  the  ISO  definition  of  the  term,  but  is  retained  at  present  to 
maintain  consistency  with  other  IGES/PDES  documents.  The  conflict  will  be  resolved  and 
documented,  but  until  then  the  term  should  be  used  with  caution. 


8.3  Acronyms 

AVM  Application  Validation  Methodology  - A test  method,  also  associated  with  a commit- 
tee of  the  IGES/PDES  Testing  Project. 

CAE  Computer  Aided  Engineering  - Used  within  this  document  to  indicate  the  sum  of  all 
activities  which  use  computers  to  assist  with  the  process  of  engineering  and  manufac- 
turing. This  would  include  design  (CAD),  manufacturing  (CAM),  analysis  (CAA  or 
FEA)  and  integrated  manufacturing  (CIM). 

GKS  Graphics  Kernel  System  - An  ISO  standard  for  application  independent  computer 
graphics  systems,  with  facilities  for  generating  pictures,  obtaining  input  from  the  user, 
and  archiving. 

IGES  Initial  Graphics  Exchange  Specification  - A specification  for  exchanging  product  data 
between  CAE  systems.  Appears  in  version  1.0,  2.0,  3.0,  4.0  and  also  as  ANSI  Y14.26M. 

ISO  International  Organization  for  Standardization  - The  organization  responsible  for  the 
development  of  international  standards  including  STEP. 

ITM  Interoperability  Testing  Methodology  - A test  method,  also  associated  with  a commit- 
tee of  the  IGES/PDES  Testing  Project. 
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OSI  Open  Systems  Interconnection  - The  ISO  series  of  standards  covering  computer  net- 
working technology. 

PDES  Product  Data  Exchange  Specification  - The  US  activity  towards  the  development  of 
STEP. 

SET  Standard  d’Echange  et  de  Transfert  - The  French  standard  (AFNOR  Z68300)  for  data 
exchange  and  archiving. 

STEP  Standard  for  the  Exchange  of  Product  Model  Data  - The  international  standard 
currently  being  developed  by  the  ISO  TC184/SC4  committee. 

TCD  Test  Case  Development  - A component  of  most  test  methods,  also  associated  with  a 
committee  of  the  IGES/PDES  Testing  Project. 

VDA-FS  - VDA-Flachen-Schnittstelle  - The  German  standard  (DIN  66301)  for  the  exchange 
of  curve  and  surface  data. 

VTM  Verification  Testing  Methodology  - A test  method,  also  associated  with  a committee 
of  the  IGES/PDES  Testing  Project. 
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9 Contact  addresses 


For  further  information  on  the  topics  discussed  in  this  document,  and  the  IGES/PDES  Testing 
Project  please  contact  : 

Mr  Mark  Pearson, 

IGES/PDES  Testing  Project  Manager 

CAD/CAM  Data  Exchange  Technical  Centre  (CADDETC) 

University  of  Leeds 
171,  Woodhouse  Lane 
Leeds  LS3  2AR 
U.K. 

Tel.  +44  532  334455  Fax  +44  532  445270 


For  information  on  the  IGES/PDES  organization  and  the  technical  committees  within  it 
contact  : 

Chairman  IGES/PDES  Organization 
National  Institute  of  Standards  & Technology 
Building  220,  Room  A150 
Gaithersburg 
MD  20899. 


For  information  on  Test  cases  within  the  public  domain  please  contact  : 

IGES/PDES  Test  Case  Librarian 

National  Institute  of  Standards  & Technology 

Building  220,  Room  A150 

Gaithersburg 

MD  20899. 
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